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Sir: 



This brief follows the Notice of Appeal filed October 13, 2008 in the above-captioned 
case and supports an appeal of the Examiner's rejection of claims in the Office Action dated July 
11,2008. 

It is requested, as suggested by the Examiner on page 2 of the 07/1 1/2008 Office Action, 
that the $255 appeal brief fee previously paid 04/21/2008 be applied to this new appeal brief, 
leaving a $15 balance due. The Commissioner is hereby authorized to charge the $15 balance 
due for the filing of an Appeal Brief to Deposit Account 50-1438 (Sommer Barnard). 
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I. Real Party in Interest 

The real parties in interest are the inventors: Joe Said and David Schleppenbach. 
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II. Related Appeals and Interferences 

Appellants, Appellants' legal representative, and Assignee know of no other prior or 
pending appeals, interferences or judicial proceedings that may be related to, directly affect or be 
directly affected by or have a bearing on the Board's decision in the pending appeal. 
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III. Status of Claims 

Pending: Claims 1-4, 6-12 and 14-22 
Canceled: 5 and 13 
Allowed: None 

Rejected: Claims 1-4, 6-12 and 14-22 
Appealed: Claims 1-4, 6-12 and 14-22 
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IV. Status of Amendments 

An amendment submitted on August 21, 2007 prior to the issuance of final rejection of 
Claims 1-4, 6-12 and 14-22 was entered by the Examiner. An Amendment After Final was filed 
on January 22, 2008 and was entered by the Examiner. No further amendments have been 
submitted. 
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V. Summary of Claimed Subject Matter 

The following explanation of the subject matter defined in each of the independent claims 
at issue in this appeal is provided with reference to page and line numbers in the specification 
and to reference characters in the drawings in compliance with 37 C.F.R. §41.37(c)(v). These 
references are made to exemplary embodiments disclosed in the application in an effort to enable 
the Board to more quickly determine where the claimed subject matter is described in the 
application, and do not limit the scope of the claims. 

A. Independent Claim 1 

The subject matter defined in independent claim 1 relates to a method of input conversion 
to output, including entering a first stage input 102 (FIG. 1) and a second input 104 (paragraph 
[0056] of the present publication). The first input and the second input are translated into an 
electronic format 106 (paragraph [0057]). The electronic format is converted into standard XML 
108 encoded with accessibility information (paragraph [0060]). The standard XML encoded 
with accessibility information is transformed into individual version of XML 110 dependent on 
desired output (paragraphs [0061] and [0062]). A rendering engine 1 12 is utilized to modify the 
individual version of XML into a format for the output (paragraph [0063]). 

B. Independent Claim 19 

The subject matter defined in independent claim 19 relates to a method of input 
conversion to output, including entering a first input 102 (FIG. 1) including verbal information 
and a second input 104 including non-verbal information (paragraphs [0056] and [0060] of the 
present publication). The second input is entered in parallel with the first input and is entered via 
a same medium as the first input. The same medium being one of a visual medium and an audial 
medium (paragraph [0056]). The first input and the second input are translated into an electronic 
format 106 (paragraph [0057]). The electronic format is converted into standard XML 108 
encoded with accessibility information (paragraph [0060]). The standard XML encoded with 
accessibility information is transformed using extensible stylesheet language transformations 
into XHTML 1 10 for desired output of text (paragraphs [0061] and [0062]). A rendering engine 
1 12 of accessible instant messenger is utilized to modify the XHTML into the desired output of 
text (paragraphs [0063 and [0064]). 

C. Independent Claim 20 

The subject matter defined in independent claim 20 relates to a method of input 
conversion to output, including entering a first input 102 (FIG. 1) including text information and 
a second input 104 including semantic information (paragraphs [0056] and [0059] of the present 
publication). The first input and the second input are translated into an electronic format 106 
(paragraph [0057]). The electronic format is converted into standard XML 108 encoded with 
accessibility information (paragraph [0060]). The standard XML encoded with accessibility 
information is transformed into individual version of XML 110 dependent on desired output 
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(paragraphs [0061] and [0062]). A rendering engine 1 12 is utilized to modify the individual 
version of XML into a format for the output (paragraph [0063]). 



8 



10/686,127 



Appeal Brief 



VI. Grounds of Rejection to be Reviewed on Appeal 

A. Claims 1-4, 6-12 and 14-22 

Claims 1-4, 6-12 and 14-22 stand rejected under 35 U.S.C. §103(a) as being unpatentable 
over U.S. Patent No. 5,748,974 (Johnson) in view of W3C ("Natural Language Semantics 
Markup Language for the Speech Interface Framework" 2000). 
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Claims 1-4, 6-12 and 14-22 were rejected under 35 U.S.C. §103(a) as being unpatentable 
over U.S. Patent No. 5,748,974 (Johnson) in view of W3C ("Natural Language Semantics 
Markup Language for the Speech Interface Framework" 2000). 

1. Independent Claim 1 is Patentable 

Claim 1 recites in part "entering a first input and a second input; translating the first input 
and the second input into an electronic format; converting the electronic format into standard 
XML encoded with accessibility information ". (Emphasis added). Appellants respectfully 
submit that such limitations are neither taught, disclosed nor suggested by the cited references 
and include distinct advantages thereover. 

The Examiner cites column 3, line 63 through column 5, line 32 of Johnson as disclosing 
"determining verbal and non-verbal semantic information (accessibility information) from the 
input and converting that into an individual version dependent on the desire output." However, 
Appellants submit that Johnson does not in fact disclose determining non-verbal semantic 
information from input. Johnson discloses "the use of multimodal (spoken, typed, handwritten) 
natural language input supplied by the user" (column 2, lines 21-23). This multimodal input is 
parsed and subjected to semantic interpretation to determine the request implicit in the parse 
(column 2, lines 8-1 1). The semantic interpretation is performed by a dialog manager 49 (FIG. 2) 
with the aid of grammars and dictionaries on direct access storage devices (DASD) 47 and 48 
(column 4, lines 4-7). The semantic interpretations of Johnson do not produce accessibility 
information . Accessibility information is non-textual, semantic information that indicates non- 
verbal aspects of the textual communication, such as tone of voice and emotional content, for 
example (paragraphs [0023] and [0060] of the present publication). What Johnson refers to as 
"semantic interpretation" is entirely verbal and is unrelated to non-verbal aspects of any 
particular text communication. That is, Johnson is completely silent as to any of the natural 
language input modes, such as handwriting, speech and typing, being used to communicate 
accessibility information. Johnson is completely silent as to providing indications of non-verbal 
aspects of the textual communication, such as tone of voice and emotional content, for example. 
Thus, Johnson does not disclose the use of accessibility information, as recited by claim 1. 

At page 2 of the Final Office Action dated 1 1/20/2007, the Examiner states that: 

[i]n response to Appellant's argument that the references fail to show certain 
features of Appellant's invention, it is noted that the features upon which 
Appellant relies (i.e., that Accessibility information is non-textual, semantic 
information that indicates non-verbal aspects of the textual communication, such 
as tone of voice and emotional content, for example (Remarks page 9)) are not 
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recited in the rejected claim(s). Although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. 

Appellants respectfully submit, however, that Appellants' definition of "accessibility 
information" is not an additional limitation from the specification that is being read into the 
claims, but rather is simply a definition of a claim term that is consistent with how the term is 
used in the specification. "An Appellant is entitled to be his or her own lexicographer and may 
rebut the presumption that claim terms are to be given their ordinary and customary meaning by 
clearing setting forth a definition of the term that is different from its ordinary and customary 
meaning(s)." MPEP §2111.01 IV., quoting In re Paulsen, 30 F.3d 1475, 1489, 31 USPQ2d 1671, 
1674 (Fed. Cir. 1994). Appellants submit that the term "accessibility information" does not have 
any ordinary or customary meaning either in the art or in common usage, and thus there is no 
presumed definition to be rebutted. 

Further, "[t]he specification should also be relied on for more than just explicit 
lexicography or clear disavowal of claim scope to determine the meaning of a claim term when 
Appellant acts as his or her own lexicographer; the meaning of a particular claim term may be 
defined by implication, that is, according to the usage of the term in the context in the 
specification." Id., quoting Phillips v. AWH Corp., 415 F. 3d 1303, 75 USPQ2d 1321 (Fed. Cir. 
2005) (en banc). Appellants submit that Appellants' proposed definition of "accessibility 
information" (i.e., non-textual, semantic information that indicates non-verbal aspects of the 
textual communication, such as tone of voice and emotional content, for example) is at least 
implicitly defined in paragraphs [0023] and [0060] of the present publication, and thus should be 
accepted. 

For all of the above reasons, Johnson does not disclose or suggest entering a first input 
and a second input, translating the first input and the second input into an electronic format, and 
converting the electronic format into standard XML encoded with accessibility information, as 
recited by claim 1. 

W3C, similarly to Johnson, discloses providing "semantic interpretations for a variety of 
inputs, including . . . speech and natural language text input." (1. Introduction, first paragraph). 
The semantic interpretations of W3C, like those of Johnson, do not produce accessibility 
information . For all of the same reasons given above with regard to Johnson, W3C does not 
disclose or suggest entering a first input and a second input, translating the first input and the 
second input into an electronic format, and converting the electronic format into standard XML 
encoded with accessibility information, as recited by claim 1 . 

The present invention includes distinct advantages over the cited references. By 
translating two inputs into an electronic format and converting the electronic format into 
standard XML encoded with accessibility information, it is possible for users with disabilities to 
communicate non-verbal aspects of textual messages. For example, one of the inputs may 
include textual information, and the other input may include information pertaining to tone of 
voice, emotional content, points of emphasis, etc. 
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For all of the above reason, claims 1, and claims 2-4, 6-12 and 14-17 depending 
therefrom, are in condition for allowance. Claims 5 and 13 were canceled in the Amendment 
filed August 21, 2007. 

2. Claim 3 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 3 is separately 
patentable. Claim 3 recites in part that the "entering step includes gesturing ." (Emphasis added). 
Appellants respectfully submit that such limitations are neither taught, disclosed nor suggested 
by the cited references and include distinct advantages thereover. 

As the Examiner acknowledges, neither Johnson nor W3C disclose that an entering step 
includes gesturing. The Examiner cites Johnson as disclosing that the input to the system can 
include any modality. However, Johnson does not suggest that an entering step includes 
gesturing. Thus, the cited references do not disclose or suggest that the entering step includes 
gesturing, as recited by claim 3. For the above reasons, claim 3 is in condition for allowance. 

3. Claim 6 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 6 is separately 
patentable. Claim 6 recites in part that the "entering step includes semantic information." 
(Emphasis added). Appellants respectfully submit that such limitations are neither taught, 
disclosed nor suggested by the cited references and include distinct advantages thereover. 

The Examiner cites Johnson as disclosing that an entering step includes semantic 
information. However, Johnson is completely silent as to entered inputs including semantic 
information. Semantic information, as defined by the present publication (paragraphs [0059] and 
[0060]), is non-verbal information that may add emotional content or emphasis to verbal content, 
for example. The American Heritage Dictionary defines "semantic" as "[o]f or relating to 
meaning, especially meaning in language." Thus, semantic information may enhance the 
meaning of verbal content. For example, a blind user could speak the word "smiley- face" to 
embed a smiley-face emoticon (paragraph [0059]). Thus, Johnson does not disclose or suggest 
an entering step including semantic information, as recited by claim 6. For the above reasons, 
claim 6 is in condition for allowance. 

4. Claim 7 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 7 is separately 
patentable. Claim 7 recites in part that the "entering step includes providing formatting and 
structure for text ." (Emphasis added). Such subject matter is disclosed at paragraph [0014] of 
the present specification. Appellants respectfully submit that such limitations are neither taught, 
disclosed nor suggested by the cited references and include distinct advantages thereover. 
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The Examiner cites Johnson as disclosing that a response from the system includes text 
which is pasted into the current application. Appellants submit that a response from the system 
is not entering inputs by a user. The Examiner, with regard to claim 1, cited speech and 
keyboard input by a user in Johnson as disclosing the entering step. Nor is pasting text into an 
application the same as providing formatting and structure for text. Thus, Johnson does not 
disclose or suggest that an entering step includes providing formatting and structure for text, as 
recited by claim 7. For the above reasons, claim 7 is in condition for allowance. 

5. Claim 8 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 8 is separately 
patentable. Claim 8 recites in part that the "translating step includes the use of automatic speech 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." 
(Emphasis added). Such subject matter is disclosed at paragraphs [0059] and [0060] of the 
present publication. Appellants respectfully submit that such limitations are neither taught, 
disclosed nor suggested by the cited references and include distinct advantages thereover. 

The Examiner cites Johnson as disclosing a first input including verbal information and a 
second input including non-verbal information. However, Johnson is completely silent as to a 
first spoken input including verbal information and a second spoken input including non-verbal 
information. Thus, Johnson does not disclose or suggest that a translating step includes the use 
of automatic speech recognition of a first input and a second input to produce an electronic 
format, the first input including verbal information, and the second input including non-verbal 
information, as recited by claim 8. For the above reasons, claim 8 is in condition for allowance. 

6. Claim 9 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 9 is separately 
patentable. Claim 9 recites in part that the "translating step includes the use of sign language 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." 
(Emphasis added). Such subject matter is disclosed at paragraphs [0059] and [0060] of the 
present publication. Appellants respectfully submit that such limitations are neither taught, 
disclosed nor suggested by the cited references and include distinct advantages thereover. 

As the Examiner acknowledges, neither Johnson nor W3C disclose that a translating step 
includes the use of sign language recognition. The Examiner cites Johnson as disclosing that the 
input to the system can include any modality. However, Johnson does not suggest that a 
translating step includes the use of sign language recognition, as recited by claim 9. Moreover, 
Johnson is completely silent as to a first gestured input including verbal information and a 
second gestured input including non-verbal information. Thus, the cited references not teach, 
disclose or suggest that a translating step includes the use of sign language recognition of a first 
input and a second input to produce an electronic format, the first input including verbal 
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information, and the second input including non-verbal information, as recited by claim 9. For 
the above reasons, claim 9 is in condition for allowance. 

7. Independent Claim 19 is Patentable 

Claim 19 recites in part "entering a first input including verbal information and a second 
input including non-verbal information, the second input being entered in parallel with the first 
input, the second input being entered via a same medium as the first input, the same medium 
being one of a visual medium and an audial medium ". (Emphasis added). A deaf user, for 
example, could enter the verbal and non-verbal inputs in parallel via a same visual medium such 
as sign language recognition. As another example, a blind user could enter the verbal and non- 
verbal inputs in parallel via a same audial medium such as automatic speech recognition. 

Johnson is completely silent as to verbal and non-verbal inputs being entered via a same 
visual or audial medium. The Examiner cites Johnson as disclosing a system accepting 
"multimodal input that is either typed or handwritten along with non-speech input that is entered 
through mouse or touch screen." However, Appellants respectfully submit that typing is not a 
same medium as a mouse or touchscreen, and handwriting is not a same medium as a mouse or 
touchscreen, as required by claim 19. Moreover, none of the mediums of typing, handwriting, a 
mouse or a touch screen is either a visual medium or an audial medium, as also is required by 
claim 19. That is, the system may receive inputs from any one of these mediums in the absence 
of light and sound, unlike a visual medium such as sign language, and unlike an audial medium 
such as speech. 

W3C is also completely silent as to verbal and non-verbal inputs being entered via a same 
visual or audial medium. Thus, the cited references do not disclose or suggest a second input 
being entered in parallel with a first input, the second input being entered via a same medium as 
the first input, the same medium being a visual medium or an audial medium, as recited by claim 
19. 

The present invention as recited by claim 19 includes distinct advantages over the cited 
references. A user with a disability may be able to provide inputs via only a single type of 
medium. For example, a blind user may be able to enter inputs though only an audial medium 
such as speech recognition. Thus, the present invention enables disabled users to enter both 
verbal and non-verbal information in parallel (i.e., as the user conceptualizes the verbal and non- 
verbal information in conjunction with each other) via a same single medium that may be the 
only medium that the disabled user may effectively use. 

Moreover, claim 19 also recites in part "converting the electronic format into standard 
XML encoded with accessibility information ". (Emphasis added). Thus, claim 19 also recites 
subject matter substantially similar to the subject matter recited by claim 1. Hence, claim 19 is 
also in condition for allowance for all of the reasons given above with regard to claim 1. 
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For the above reasons, claim 19, and claims 21 and 22 depending therefrom, are in 
condition for allowance. 

8. Claim 21 is Separately Patentable 

Claim 21, depending from claim 19, recites that "the same medium comprises automatic 
speech recognition ." (Emphasis added). Thus, both verbal and non-verbal information are 
entered in parallel via speech recognition. The Examiner cites Johnson as disclosing a system 
receiving speech and interpreting gestures. Although Appellants respectfully submit that 
Johnson does not suggest interpreting gestures, Appellants further submit that gesture 
interpretation is not automatic speech recognition, as required by claim 21. Thus, Appellants 
submit that claim 21 is in condition for allowance. 

9. Claim 22 is Separately Patentable 

Claim 22, depending from claim 19, recites that "the same medium comprises sign 
language recognition ." (Emphasis added). Thus, both verbal and non-verbal information are 
entered in parallel via sign language recognition. Appellants submit that neither of the cited 
references disposes or suggests sign language recognition, and neither reference discloses 
inputting verbal and non-verbal information via sign language recognition, as required by claim 
22. Appellants submit that claim 22 is in condition for allowance. 

10. Independent Claim 20 is Patentable 

Claim 20 recites "entering a first input including text information and a second input 
including semantic information ; translating the first input and the second input into an electronic 
format; converting the electronic format into standard XML encoded with accessibility 
information ." (Emphasis added). The Examiner cites Johnson as disclosing entering a first input 
including text information and a second input including semantic information. However, 
Appellants respectfully submit that Johnson does not disclose entering input including semantic 
(non-verbal) information as "semantic information" is defined in paragraphs [0059] and [0060] 
of the present publication. Semantic information, as defined by the present publication 
(paragraphs [0059] and [0060]), is non-verbal information that may add emotional content or 
emphasis to verbal content, for example. The American Heritage Dictionary defines "semantic" 
as "[o]f or relating to meaning, especially meaning in language." Thus, semantic information 
may enhance the meaning of verbal content. For example, a blind user could speak the word 
"smiley-face" to embed a smiley-face emoticon (paragraph [0059]). Johnson is completely silent 
as to entering an input including semantic information. As discussed above with regard to claim 
6, the semantic information may enhance the meaning of the text information with emotional 
content or points of emphasis, for example. 

Moreover, as discussed above with regard to claim 1, the cited references do not disclose 
converting an electronic format into standard XML encoded with accessibility information , as 
also required by claim 20. For the foregoing reasons, the cited references do not disclose or 
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suggest entering a first input including text information and a second input including semantic 
information ; translating the first input and the second input into an electronic format; and 
converting the electronic format into standard XML encoded with accessibility information, as 
recited by claim 20. Thus, claim 20 is in condition for allowance. 

In view of the remarks above, Appellants respectfully submit that the present invention is 
in condition for allowance and respectfully request the Honorable Board of Appeals to reverse 
the Examiner's rejections from the Office Action. 

Respectfully submitted, 



December 12. 2008 /Keith J. Swedo/ 

Date Keith J. Swedo, Esq. 

Reg. No. 43,176 
Tart Stettinius & Hollister LLP 
One Indiana Square, Suite 3500 
Indianapolis, IN 46204 
(317) 713-3500 (phone) 
(317)713-3699 (fax) 
kswedo @taftlaw. com 
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VIII. Claims Appendix 

List of Claims 

1 . A method of input conversion to output, said conversion method comprising: 
entering a first stage input and a second input; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility 
information; 

transforming the standard XML encoded with accessibility information into individual 
version of XML dependent on desired output; and 

utilizing a rendering engine to modify the individual version of XML into a format for 
the output. 

2. The conversion method of claim 1 wherein said entering step includes speaking. 

3. The conversion method of claim 1 wherein said entering step includes gesturing. 

4. The conversion method of claim 1 wherein said entering step includes writing. 

5. (Canceled) 

6. The conversion method of claim 1 wherein said entering step includes semantic information. 
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7. The conversion method of claim 1 wherein said entering step includes providing formatting 
and structure for text. 



8. The conversion method of claim 2 wherein said translating step includes the use of automatic 
speech recognition of the first input and the second input to produce the electronic format, the 
first input including verbal information, and the second input including non-verbal information. 

9. The conversion method of claim 3 wherein said translating step includes the use of sign 
language recognition of the first input and the second input to produce the electronic format, the 
first input including verbal information, and the second input including non-verbal information. 

10. The conversion method of claim 1 wherein the individual version of XML is XHTML for 
desired output. 

11. The conversion method of claim 1 wherein the individual version of XML is VoiceXML for 
desired output. 

12. The conversion method of claim 1 wherein the accessibility information includes non-verbal 
information. 

13. (Canceled) 
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14. The conversion method of claim 1 1 wherein the desired output is synthesized speech. 

15. The conversion method of claim 12 wherein the desired output is virtual sign language. 

16. The conversion method of claim 12 wherein the desired output is electronic braille. 

17. The conversion method of claim 12 wherein the desired output is electronic large print. 

18. The conversion method of claim 1 wherein said transforming step includes the use of 
extensible stylesheet language transformations to transform the standard XML encoded with 
accessibility information into the individual version of XML dependent on desired output. 
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19. A method of input conversion to output, said conversion method comprising: 

entering a first input including verbal information and a second input including non- 
verbal information, the second input being entered in parallel with the first input, the second 
input being entered via a same medium as the first input, the same medium being one of a visual 
medium and an audial medium; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility 
information; 

transforming the standard XML encoded with accessibility information using extensible 
stylesheet language transformations into XHTML for desired output of text; and 

utilizing a rendering engine of accessible instant messenger to modify the XHTML into 
the desired output of text. 
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20. A method of input conversion to output, said conversion method comprising: 

entering a first input including text information and a second input including semantic 
information; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility 
information; 

transforming the standard XML encoded with accessibility information into individual 
version of XML dependent on desired output; and 

utilizing a rendering engine to modify the individual version of XML into a format for 
the output. 

21. The conversion method of claim 19 wherein the same medium comprises automatic speech 
recognition. 

22. The conversion method of claim 19 wherein the same medium comprises sign language 
recognition. 
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IX. Evidence Appendix 

No evidence has been entered. 
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X. Related Proceedings Appendix 

Appellants are unaware of any Related Proceedings. 
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